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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
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Filed : 11/13/2001 

Confirmation : 2739 
TC/A.U. : 2424 
Examiner : BAIG, Sahar A. 
Atty. Docket : US010588US 

Title: AFFECTIVE TELEVISION MONITORING AND CONTROL 

Mail Stop: APPEAL BRIEF - PATENTS 
Commissioner for Patents 
Alexandria, VA 22313-1450 

REPLY BRIEF UNDER 37 CFR 41 .41 

Sir: 

This is a Reply Brief in response to the Examiner's answer dated 18 June 
2010 in the subject application. 

RESTATEMENT OF GROUNDS OF REJECTION 
Claims 1, 3-8\ 10-30, 34^, 36, and 39-40 stand rejected under 35 U.S.C. 

103(a) over Zawilinski (USP 5,676,138) in view of Hoffberg et al. (USP 6,400,996, 

hereinafter Hoffberg) and Strubbe (USP 5,483,278). 

Claims 31-32 stand rejected under 35 U.S.C. 103(a) over Zawilinski in view of 

Hoffberg, Strubbe, and Black et al. (USP 5,774,591, hereinafter Black). 

Claims 33 and 38 stand rejected under 35 U.S.C. 103(a) over Zawilinski in 

view of Hoffberg, Strubbe, and Bentolila et al. (USPA 2003/0101449, hereinafter 

Bentolila). 

Claim 37 stands rejected under 35 U.S.C. 103(a) over Zawilinski in view of 

Hoffberg, Strubbe, and Shinohara (USPA 2003/0005431 ). 

^ The Office action and tine Examiner's Answer state that claims 1-8 are rejected; however, claim 2 has 
been canceled. 

^ The Office action and the Examiner's Answer omit claim 34 in the statement of rejection. The 
applicants believe that this is a typographical error because the Examiner subsequently references 
claim 34 as being "encompassed within the rejection of Claim 1". 
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REIVIARKS REGARDING EXAMINER'S ANSWER 
Claims 1,3-8, 10-34, 36-40 

The Examiner asserts that Zawilinski discloses descriptive information 
related to the stimuli that produces a user reaction at column 5, lines 35-65. This 
assertion is incorrect. 

At the cited text, Zawilinski discloses: 

"field of research regarding emotional responses of human beings, more particularly, 
a system including measurement devices of physiological variables operably 
connected to a computerized analyzer and having a multimedia display for 
displaying resuhing data, such as each of a plurality of semantic descriptors 
associated with each of a plurality of emotional responses of a human at a given 
moment over a preselected time period during the presentation of a stimulus. 

As a matter of background, the present invention operates on two theories of 
emotion, each of which are based on the premise that one or more physiological 
responses, such as electrical muscle activity, precede the identification of an 
emotional response in reaction by an individual to a stimulus. 

The present invention comprises (1) a stimulus presentation device for presenting 
a stimulus occurring over a predetermined period of time to each of one or more 
individuals forming a population sample, measviring devices to measure a change in 
each of a plurality of physiological variables, (2) translation or processing means, 
namely a software program installed on a computerized processing means, to 
translate each value of a physiological variable for each individual into a common 
unitless measure of emotional response at any one preselected unit of time over a 
preselected period of time and to apply automatically a semantic descriptor to each 
common unitless measure of emotional response falling on a sector of an interaction 
index, and (3) an interactive multimedia computer for electronically storing, 
displaying and retrieving information and capable of at least visually displaying at 
least one stimulus" 

As is clearly evident, nowhere in the cited text does Zawilinski address descriptive 
information that is related to the stimuli that produces the user's reaction, and in 
particular, nowhere in the cited text does Zawilinski disclose a processor for receiving 
and analyzing a signal representative of the physical reaction of a viewer to 
determine if it can be associated with at least one recognizable viewer emotional 
response, whereupon the processor associates the recognized emotional 
response with a descriptive information relating to a program that was being 
displayed when the physical reaction was sensed, to provide a viewer preference 
relating to the descriptive Information, as specifically claimed in claim 1, upon 
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which claims 3-8, 10, and 40 depend. The Examiner relies on the rejection of claim 1 
to support the rejection of claims 11-30, 34, 36, and 39 (Examiner's Answer, page 17, 
paragraph b). The Examiner also relies on the rejection of claim 1 1 to support the 
rejection of claims 31-33 and 37-38 (Examiner's Answer, pages 19 and 20, 
paragraphs g through i.) 

Zawilinski discloses determining a semantic descriptor corresponding to an 
emotional response of a viewer to an advertisement. The Examiner apparently 
considers the fact that this semantic descriptor is used to assess the 
program/advertisement being viewed corresponds to the claimed descriptive 
information relating to the program being viewed. This interpretation is inconsistent 
with the applicants' claim. 

The applicants' claim 1 includes a two-step process. In a first step, the 
physical reaction of a viewer is analyzed to determine a recognized emotional 
response. Then, in a second step, this recognized emotional response is associated 
with descriptive information relating to a program. For example, based on the 
viewer's physical reaction, the program may determine that the viewer is exhibiting 
laughter. Then, in a second step, the program may associate laughter with 
descriptive information related to the program, such as the actor, the director, the 
title, the genre, and so on. In this way, for example, the program can determine that a 
particular actor generally evokes laughter from the viewer. Zawilski does not perform 
this second step; Zawilski is silent with respect to descriptive information related to 
the stimuli that evoked the monitored response. 

The applicants respectfully maintain that Zawilinski's semantic descriptor 
corresponds to the applicants' claimed recognized emotional response (e.g. 
laughter). The applicants note that the Examiner's analysis of claim 1 supports this 
correspondence. The Examiner states that Zawilinski discloses analyzing the 
physical reaction of a viewer to determine an associated recognized emotional 
response at column 4, lines 47-57. At the cited text, Zawilinski discloses: 
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"Accordingly, it is a principal object of the invention to provide a means by which 
to detect a human emotional response at a moment in time in response to stimulus 
presented at that moment in time. 

It is another object of the invention to provide means by which an emotional 
response can be detected by directly measuring an associated physiological variable. 

It is a further object of the invention to provide means by which an emotional 
response can be semantically described automatically upon detection." (Zawilinski, 
column 4, lines 47-57). 

Given that the Examiner cites Zawiiinski's determination of a semantic description to 
read upon the applicants' determination of a recognizable viewer emotional 
response, the applicants respectfully maintain that the Examiner's subsequent 
assertion that this same determination of the semantic description corresponds to a 
determination that associates the semantic description with descriptive information 
relating to a program is unfounded and/or contradictory. 

The Examiner's redundant use of Zawiiinski's determination of a semantic 
descriptor as corresponding to each of the two-step process claimed by the 
applicants distorts the meaning of the applicants' claimed invention. If the descriptive 
information relating to a program is considered to be this same semantic description, 
then the second step of the applicants' claim would be interpreted to correspond to: 
"whereupon the processor associates the [semantic descriptor of the user reaction] 
with a [semantic descriptor of the user reaction] relating to a program that was being 
displayed when the physical reaction was sensed", which obviously renders this 
second limitation meaningless. 

Additionally, claim 1 recites that the association of the recognized emotional 
response with descriptive information relating to a program is performed to provide a 
viewer preference relating to tiie descriptive information. If Zawiiinski's semantic 
descriptor of the user's reaction (happy, sad, content, agitated, etc.) is interpreted to 
correspond to the descriptive information relating to a program, and descriptive 
information of the program (comedy, drama, news, includes John Wayne, etc.) is not 
provided, such a determination of the users preference with regard to the descriptive 
information related to the program can not be made. 
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The Examiner's equating of Zawilinski's semantic descriptor to both the 
determined emotional response and the descriptive information would result in the 
determined viewer preference being something like "the user is {happy, sad, content, 
agitated, etc.} when the program makes the user {happy, sad, content, agitated, 
etc.}". Without an emotional response [semantic descriptor] being distinguished from 
the descriptive information related to the program, the Examiner's interpretation of 
Zawilinski renders this further aspect of the applicants' claim meaningless 

Contrarily, as noted above, using a reasonable and meaningful interpretation 
of the applicants' claim in the context of the application, wherein the emotional 
response {happy, sad, content, agitated, etc.} is distinguished from the descriptive 
information {comedy, drama, news, includes John Wayne, etc.}, the determined 
viewer's preference would be something like "the user is {happy, sad, content, 
agitated, etc.} when the program being viewed is {comedy, drama, news, includes 
John Wayne, etc.}", because such an association provides a meaningful and useful 
determination of the user's preference for the descriptive information related to the 
program, as claimed. 

Because Zawilinski fails to teach or suggest associating a recognized 
emotional response with a descriptive information relating to a program to provide a 
viewer preference relating to the descriptive information, as specifically claimed by 
the applicants, the applicants respectfully maintain that the Examiner has failed to 
establish a prima facie case to support the rejections of claims 1, 3-8, 10-34, and 36- 
40 under 35 U.S.C. 103(a) that rely on Zawilinski for this teaching. Accordingly, the 
applicants respectfully request that these rejections be reversed by the Board. 

The Examiner also asserts that Hoffberg discloses associating a recognized 
emotional response or mood with descriptive information relating to a program to 
identify a user's preference at column 65, lines 33-55. This assertion is also incorrect. 

At the cited text, Hoffberg discloses: 



US010588US Reply Brief 9.0828 



Atty. Docket No. US010588US 



Appl. No. 10/014,179 
Reply Brief in Response to 
Examiner's Answer of 18 June 2010 



Page 6 of 8 



"In determining mood, a number of physiologic parameters may be detected. In a 
training circumstance, these set of parameters are correlated with a temporally 
associated preference. Thus, when a user inputs a preference into the system as 
feedback, mood data is also obtained. Invariant preferences may be separated, and 
analyzed globally, without regard for temporal variations, while varying preferences 
are linked with information regarding the surrounding circumstances and stored. For 
example, the preference data may be used to train a neural network, e.g., using 
backpropagation of errors or other known methods. The inputs to the neviral network 
include available data about surrounding context, such as time, environmental 
brightness, and persons present; source program choices, which may be raw data, 
preprocessed data, and abstracted data; explicit user input; and, in this embodiment, 
mood parameters, which may be physiological or biometric data, voice pattern, or 
implicit inputs. An example of an implicit input is an observation of a man-machine 
interaction, such as a video game. The manner in which a person plays a video game 
or otherwise interacts with a machine may provide valuable data for determining a 
mood or preference." (Hotfberg, column 65, lines 33-55.) 

As is again clearly evident, the cited text does not address descriptive information 

relating to a program, and in particular does not disclose associating a recognized 

emotional response or mood with descriptive information relating to a program to 

identify a user's preference, as asserted by the Examiner. 

In Hoffberg, the user's emotion or mood may be determined by the manner in 
which the person plays a game or interacts with the system, but Hoffberg does not 
teach or suggest that this determination of emotion or mood will be associated with 
descriptive information of the game to identify the user's preference relative to this 
descriptive information, as specifically claimed by the applicants. 

Because Hoffberg fails to teach or suggest associating a recognized emotional 
response or mood with descriptive information relating to a program to identify a 
user's preference relative to the descriptive information, as specifically claimed by the 
applicants, the applicants respectfully maintain that the Examiner has failed to 
establish a prima facie case to support the rejections of claims 1, 3-8, 10-34, and 36- 
40 under 35 U.S.C. 103(a) that rely on Hoffberg for this teaching. Accordingly, the 
applicants respectfully request that these rejections be reversed by the Board. 
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Claims 18-22 

Tine Examiner asserts that Hoffberg discloses determining a distinguishing 
characteristic of a displayed programming segment and associating a viewer 
response corresponding to a physical condition perceived during the display of the 
programming segment with a viewer preference level at column 61, lines 14-30. This 
assertion is incorrect. 

At the cited text, Hoffberg discloses: 

"The present invention also allows a dynamic user preference profile determination 
based on explicit or implicit desires, e.g., moods, which assist in processing data to 
make decisions which conform to the user preference at a given point in time. For 
example, voice patterns, skin temperature, heat pulse rate, external context, skin 
resistance (galvanic skin response), blood pressure, stress, as determined by EMG, 
EEG or other known methods, spontaneous motor activity or twitching, may be 
detected in order to determine or infer a user mood, which may be used as a dynamic 
influence on the user preference. These dynamic influences are preferably stored 
separately from static influences of the preferences, so that a resultant determined 
preference includes a dynamic influence based on a determined mood or other 
temporally varying factor and a static influence associated with the user." (Hoffberg, 
column 61, lines 14-30.) 

As is again clearly evident, the cited text does not address determining 

characteristics of a displayed program segment, and thus cannot be said to disclose 

determining a distinguishing characteristic of a displayed programming segment and 

associating a viewer response corresponding to a physical condition perceived during 

the display of the programming segment with a viewer preference level, as 

specifically claimed by the applicants. 

Because the Examiner has failed to establish a prima facie case to support the 

rejection of claims 18-22, the applicants respectfully maintain that the rejection of 

claims 18-22 under 35 U.S.C. 103(a) over Zawilinski in view of Hoffberg and Strubbe 

is unfounded, and should be reversed by the Board. 
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CONCLUSIONS 

Because the combination of Zawilinski, Hoffberg, and Strubbe fails to teach or 
suggest associating a recognized emotional response with a descriptive information 
relating to a program that was being displayed when a physical reaction was sensed, 
to provide a viewer preference relating to the descriptive information, the applicants 
respectfully request that the Examiner's rejection of claims 1, 3-8, 10-34, and 36-40 
under 35 U.S.C. 103(a) be reversed by the Board, and the claims be allowed to pass 
to issue. 

Because the combination of Zawilinski, Hoffberg, and Strubbe fails to teach or 
suggest determining a distinguishing characteristic of a displayed programming 
segment and associating a viewer response corresponding to a physical condition 
perceived during the display of the programming segment with a viewer preference 
level, the applicants respectfully request that the Examiner's rejection of claims 18-22 
under 35 U.S.C. 103(a) be reversed by the Board, and the claims be allowed to pass 
to issue. 

Respectfully submitted, 

/Robert M. McDermott/ 
Robert M. McDermott, Esq. 
Registration Number 41,508 
Phone: 804-493-0707 
for: Kevin C. Ecker 
Reg. 43,600 
914-333-9618 



Please direct all correspondence to: 

Corporate Counsel 

Philips Intellectual Property and Standards 
P.O. Box 3001 

Briarcliff Manor, NY 10510-8001 
Phone: (914)333-9618 
Fax: (914)332-0615 
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